129 research outputs found
A Quasi-Wasserstein Loss for Learning Graph Neural Networks
When learning graph neural networks (GNNs) in node-level prediction tasks,
most existing loss functions are applied for each node independently, even if
node embeddings and their labels are non-i.i.d. because of their graph
structures. To eliminate such inconsistency, in this study we propose a novel
Quasi-Wasserstein (QW) loss with the help of the optimal transport defined on
graphs, leading to new learning and prediction paradigms of GNNs. In
particular, we design a "Quasi-Wasserstein" distance between the observed
multi-dimensional node labels and their estimations, optimizing the label
transport defined on graph edges. The estimations are parameterized by a GNN in
which the optimal label transport may determine the graph edge weights
optionally. By reformulating the strict constraint of the label transport to a
Bregman divergence-based regularizer, we obtain the proposed Quasi-Wasserstein
loss associated with two efficient solvers learning the GNN together with
optimal label transport. When predicting node labels, our model combines the
output of the GNN with the residual component provided by the optimal label
transport, leading to a new transductive prediction paradigm. Experiments show
that the proposed QW loss applies to various GNNs and helps to improve their
performance in node-level classification and regression tasks
Regularized Optimal Transport Layers for Generalized Global Pooling Operations
Global pooling is one of the most significant operations in many machine
learning models and tasks, which works for information fusion and structured
data (like sets and graphs) representation. However, without solid mathematical
fundamentals, its practical implementations often depend on empirical
mechanisms and thus lead to sub-optimal, even unsatisfactory performance. In
this work, we develop a novel and generalized global pooling framework through
the lens of optimal transport. The proposed framework is interpretable from the
perspective of expectation-maximization. Essentially, it aims at learning an
optimal transport across sample indices and feature dimensions, making the
corresponding pooling operation maximize the conditional expectation of input
data. We demonstrate that most existing pooling methods are equivalent to
solving a regularized optimal transport (ROT) problem with different
specializations, and more sophisticated pooling operations can be implemented
by hierarchically solving multiple ROT problems. Making the parameters of the
ROT problem learnable, we develop a family of regularized optimal transport
pooling (ROTP) layers. We implement the ROTP layers as a new kind of deep
implicit layer. Their model architectures correspond to different optimization
algorithms. We test our ROTP layers in several representative set-level machine
learning scenarios, including multi-instance learning (MIL), graph
classification, graph set representation, and image classification.
Experimental results show that applying our ROTP layers can reduce the
difficulty of the design and selection of global pooling -- our ROTP layers may
either imitate some existing global pooling methods or lead to some new pooling
layers fitting data better. The code is available at
\url{https://github.com/SDS-Lab/ROT-Pooling}
Graph Self-Contrast Representation Learning
Graph contrastive learning (GCL) has recently emerged as a promising approach
for graph representation learning. Some existing methods adopt the 1-vs-K
scheme to construct one positive and K negative samples for each graph, but it
is difficult to set K. For those methods that do not use negative samples, it
is often necessary to add additional strategies to avoid model collapse, which
could only alleviate the problem to some extent. All these drawbacks will
undoubtedly have an adverse impact on the generalizability and efficiency of
the model. In this paper, to address these issues, we propose a novel graph
self-contrast framework GraphSC, which only uses one positive and one negative
sample, and chooses triplet loss as the objective. Specifically, self-contrast
has two implications. First, GraphSC generates both positive and negative views
of a graph sample from the graph itself via graph augmentation functions of
various intensities, and use them for self-contrast. Second, GraphSC uses
Hilbert-Schmidt Independence Criterion (HSIC) to factorize the representations
into multiple factors and proposes a masked self-contrast mechanism to better
separate positive and negative samples. Further, Since the triplet loss only
optimizes the relative distance between the anchor and its positive/negative
samples, it is difficult to ensure the absolute distance between the anchor and
positive sample. Therefore, we explicitly reduced the absolute distance between
the anchor and positive sample to accelerate convergence. Finally, we conduct
extensive experiments to evaluate the performance of GraphSC against 19 other
state-of-the-art methods in both unsupervised and transfer learning settings.Comment: ICDM 2023(Regular
Joint Port Selection Based Channel Acquisition for FDD Cell-Free Massive MIMO
In frequency division duplexing (FDD) cell-free massive MIMO, the acquisition
of the channel state information (CSI) is very challenging because of the large
overhead required for the training and feedback of the downlink channels of
multiple cooperating base stations (BSs). In this paper, for systems with
partial uplink-downlink channel reciprocity, and a general spatial domain
channel model with variations in the average port power and correlation among
port coefficients, we propose a joint-port-selection-based CSI acquisition and
feedback scheme for the downlink transmission with zero-forcing precoding. The
scheme uses an eigenvalue-decomposition-based transformation to reduce the
feedback overhead by exploring the port correlation. We derive the sum-rate of
the system for any port selection. Based on the sum-rate result, we propose a
low-complexity greedy-search-based joint port selection (GS-JPS) algorithm.
Moreover, to adapt to fast time-varying scenarios, a supervised deep
learning-enhanced joint port selection (DL-JPS) algorithm is proposed.
Simulations verify the effectiveness of our proposed schemes and their
advantage over existing port-selection channel acquisition schemes.Comment: 30 pages, 9 figures. The paper has been submitted to IEEE journal for
possible publicatio
Wirelessly-Controlled Untethered Piezoelectric Planar Soft Robot Capable of Bidirectional Crawling and Rotation
Electrostatic actuators provide a promising approach to creating soft robotic
sheets, due to their flexible form factor, modular integration, and fast
response speed. However, their control requires kilo-Volt signals and
understanding of complex dynamics resulting from force interactions by on-board
and environmental effects. In this work, we demonstrate an untethered planar
five-actuator piezoelectric robot powered by batteries and on-board
high-voltage circuitry, and controlled through a wireless link. The scalable
fabrication approach is based on bonding different functional layers on top of
each other (steel foil substrate, actuators, flexible electronics). The robot
exhibits a range of controllable motions, including bidirectional crawling (up
to ~0.6 cm/s), turning, and in-place rotation (at ~1 degree/s). High-speed
videos and control experiments show that the richness of the motion results
from the interaction of an asymmetric mass distribution in the robot and the
associated dependence of the dynamics on the driving frequency of the
piezoelectrics. The robot's speed can reach 6 cm/s with specific payload
distribution.Comment: Accepted to the 2023 IEEE International Conference on Robotics and
Automation (ICRA
DGI: Easy and Efficient Inference for GNNs
While many systems have been developed to train Graph Neural Networks (GNNs),
efficient model inference and evaluation remain to be addressed. For instance,
using the widely adopted node-wise approach, model evaluation can account for
up to 94% of the time in the end-to-end training process due to neighbor
explosion, which means that a node accesses its multi-hop neighbors. On the
other hand, layer-wise inference avoids the neighbor explosion problem by
conducting inference layer by layer such that the nodes only need their one-hop
neighbors in each layer. However, implementing layer-wise inference requires
substantial engineering efforts because users need to manually decompose a GNN
model into layers for computation and split workload into batches to fit into
device memory. In this paper, we develop Deep Graph Inference (DGI) -- a system
for easy and efficient GNN model inference, which automatically translates the
training code of a GNN model for layer-wise execution. DGI is general for
various GNN models and different kinds of inference requests, and supports
out-of-core execution on large graphs that cannot fit in CPU memory.
Experimental results show that DGI consistently outperforms layer-wise
inference across different datasets and hardware settings, and the speedup can
be over 1,000x.Comment: 10 pages, 10 figure
Model-Based Control of Planar Piezoelectric Inchworm Soft Robot for Crawling in Constrained Environments
Soft robots have drawn significant attention recently for their ability to
achieve rich shapes when interacting with complex environments. However, their
elasticity and flexibility compared to rigid robots also pose significant
challenges for precise and robust shape control in real-time. Motivated by
their potential to operate in highly-constrained environments, as in
search-and-rescue operations, this work addresses these challenges of soft
robots by developing a model-based full-shape controller, validated and
demonstrated by experiments. A five-actuator planar soft robot was constructed
with planar piezoelectric layers bonded to a steel foil substrate, enabling
inchworm-like motion. The controller uses a soft-body continuous model for
shape planning and control, given target shapes and/or environmental
constraints, such as crawling under overhead barriers or "roof" safety lines.
An approach to background model calibrations is developed to address deviations
of actual robot shape due to material parameter variations and drift. Full
experimental shape control and optimal movement under a roof safety line are
demonstrated, where the robot maximizes its speed within the overhead
constraint. The mean-squared error between the measured and target shapes
improves from ~0.05 cm without calibration to ~0.01 cm with
calibration. Simulation-based validation is also performed with various
different roof shapes.Comment: Accepted to the 2022 IEEE 5th International Conference on Soft
Robotics (RoboSoft). Project website: https://piezorobotcontroller.github.io/
Summary video: https://youtu.be/Md-Uo-pUaI
eViper: A Scalable Platform for Untethered Modular Soft Robots
Soft robots present unique capabilities, but have been limited by the lack of
scalable technologies for construction and the complexity of algorithms for
efficient control and motion, which depend on soft-body dynamics,
high-dimensional actuation patterns, and external/on-board forces. This paper
presents scalable methods and platforms to study the impact of weight
distribution and actuation patterns on fully untethered modular soft robots. An
extendable Vibrating Intelligent Piezo-Electric Robot (eViper), together with
an open-source Simulation Framework for Electroactive Robotic Sheet (SFERS)
implemented in PyBullet, was developed as a platform to study the sophisticated
weight-locomotion interaction. By integrating the power electronics, sensors,
actuators, and batteries on-board, the eViper platform enables rapid design
iteration and evaluation of different weight distribution and control
strategies for the actuator arrays, supporting both physics-based modeling and
data-driven modeling via on-board automatic data-acquisition capabilities. We
show that SFERS can provide useful guidelines for optimizing the weight
distribution and actuation patterns of the eViper to achieve the maximum speed
or minimum cost-of-transportation (COT).Comment: 8 pages, 21 figures, accepted by IROS 202
Temperature controlled microcapsule loaded with Perilla essential oil and its application in preservation of peaches
In this study, Perilla frutescens essential oil (PEO) loaded microcapsules (PEOM) were successfully prepared and their thermal stability, temperature-responsive releasing effect, antioxidant activity, antibacterial activity, and preservation of peach were systematically investigated. PEOM showed excellent encapsulation efficiency (91.5%) with a core-shell ratio of 1.4:1 and exhibited high thermal stability, indicating that PEOM could effectively maintain PEO release rate. In vitro assays indicated that the optimal kinetic model for PEO release fitted well with first order with a diffusion mechanism. A high level of antioxidant and antibacterial activity of PEOM was maintained. In addition, owing to its sustained release, PEOM could prolong the shelf life of peaches significantly. Therefore, PEOM has potential application and development prospects in the field of food preservation
- …